Parameters to evaluate a simple linear regression model: R² and SEE

13/07/2026, 업데이트 29/10/2026 •0•0

How to evaluate a simple linear regression: residuals, SEE, t-test, confidence interval and R², with a step-by-step worked example and exercises.

작성자:Leon

번역자Editorial

개정자Editorial

In this class we will see how to evaluate whether a simple linear regression model is good: residual analysis, the standard error of the estimate (SEE), the t-test, the confidence interval and the coefficient of determination (R²). At the end, we will work through exercises with data tables to calculate regressions and interpret the results.

Simple linear regression models the relationship between a dependent variable (Y) and an independent variable (X) with a straight line, \( \hat{Y} = \beta_0 + \beta_1 X \). But how do we know whether the model is reliable? To find out, we evaluate its quality with a few statistical indicators.

Quality Indicators of a Linear Regression

Residual Analysis

The residuals are the model's errors for each observation:

\[ e_i = y_i - \hat{y}_i \]

In a good model, the residuals should:

Be random: no clear patterns when plotted against the predicted values or against X.
Have a mean of zero: no systematic bias. In least-squares regression with an intercept, the mean of the residuals is always zero by construction; what matters is checking that they are not positive in one range of X and negative in another.
Have constant variance (homoscedasticity).
Follow an approximately normal distribution, so that tests and confidence intervals are valid.

How do we assess the residuals?

Method	What to check?
Residuals vs. predicted plot	Whether there are patterns (curves, funnels)
Histogram of residuals	Whether the distribution is bell-shaped and centred at zero
Q-Q plot	Whether the points follow the normal line

Common problems:

Non-linearity: U-shaped or curved residuals indicate that the relationship between X and Y is not a straight line.
Heteroscedasticity: non-constant variance (funnel-shaped plot).
Non-normality: highly skewed or heavy-tailed residuals (affects the statistical tests).
Autocorrelation: the residual of one observation helps predict the next one (common in time series).

Example of a model whose predictions track the actual values well (here a time-series model, but the plot is read the same way):

Well-fitted regression

Example of a residual distribution:

Good residual

Most of the residuals are concentrated around zero, in a roughly symmetric way, which is a good sign. Notice, however, the few isolated values near −75 and +100: they make the tails heavier than those of the normal. This is where skewness and kurtosis come in: we want residuals that are close to symmetric (skewness near 0) and mesokurtic (excess kurtosis near 0).

Standard Error of the Estimate (SEE)

The SEE (Standard Error of the Estimate) measures the typical size of the error the model makes when predicting Y. It is the standard deviation of the residuals, in the same unit as Y:

\[ SEE = \sqrt{\frac{\sum (y_i - \hat{y}_i)^2}{n - 2}} = \sqrt{\frac{SSE}{n-2}} \]

The denominator is \( n - 2 \) because two parameters (\( \beta_0 \) and \( \beta_1 \)) were estimated from the same data.

The smaller the SEE, the more precise the predictions.
It is useful for comparing models that predict the same variable Y.

t-Test and p-Value

The t-test checks whether the slope coefficient (\( \beta_1 \)) is statistically different from zero.

Hypotheses:

\( H_0: \beta_1 = 0 \) (no linear relationship)
\( H_1: \beta_1 \neq 0 \) (there is a linear relationship)

The test statistic divides the coefficient by its standard error and has \( n - 2 \) degrees of freedom:

\[ t = \frac{\hat{\beta}_1}{SE(\hat{\beta}_1)}, \quad SE(\hat{\beta}_1) = \frac{SEE}{\sqrt{\sum (x_i - \bar{x})^2}} \]

Interpreting the p-value (at a 5% significance level):

p < 0.05: reject \( H_0 \) (significant relationship).
p ≥ 0.05: do not reject \( H_0 \) (not enough evidence of a relationship).

Confidence Interval for the Coefficients

The 95% confidence interval for \( \beta_1 \) shows the range of plausible values for the slope:

\[ \hat{\beta}_1 \pm t_{0.025;\,n-2} \cdot SE(\hat{\beta}_1) \]

If the interval does not include zero, the relationship is significant at the 5% level (the same conclusion as the t-test).

Coefficient of Determination (R²)

The R² (coefficient of determination) measures how much of the variation in Y is explained by X:

\[ R^2 = \frac{SS_{\text{Reg}}}{SS_{\text{Total}}} = 1 - \frac{SSE}{SS_{\text{Total}}} \]

Where:

\( SS_{\text{Reg}} \) = regression sum of squares (explained variation);
\( SSE \) = sum of squared residuals (unexplained variation);
\( SS_{\text{Total}} \) = total sum of squares, \( SS_{\text{Total}} = SS_{\text{Reg}} + SSE \).

Note: the second form uses the sum of squared residuals (SSE), not the SEE. In simple regression, R² is also the square of the correlation coefficient: \( R^2 = r^2 \).

Interpretation:

0 ≤ R² ≤ 1 (the closer to 1, the more of the variation in Y is explained).
If \( R^2 = 0.75 \), 75% of the variation in Y is explained by X.

Limitations of R²:

It does not indicate causality (only association).
It can be misleading if there are outliers or non-linear relationships: a high R² does not guarantee that a straight line is the right model, so it should be read together with the residual plot.
With few observations, a high R² may not be statistically significant (see exercise 3).

When analysing a regression, you see that the residuals form a U: they are positive for small and large X and negative in the middle. What does this indicate?

Answer: A non-linear relationship that the straight line did not capture — the U pattern shows that the errors depend on X in a curved way; the fix is to add a quadratic term or transform the variables. Heteroscedasticity shows up as a funnel (the spread of the residuals grows or shrinks with X), not as a U.

A regression with 12 observations has SS Total = 500 and SSE = 100. What are the R² and the SEE?

Answer: R² = 0.80 and SEE ≈ 3.16 — R² = 1 minus 100/500 = 0.80, and SEE = square root of 100/(12 minus 2) = square root of 10 ≈ 3.16. The 0.20 option is the unexplained share; the SEE = 10 option forgets the square root.

Practical Example with a Data Table

Let's analyse a regression between study hours (X) and exam score (Y).

Student	Hours (X)	Score (Y)
1	2	50
2	4	60
3	6	80
4	8	70
5	10	90

Steps to evaluate the regression:

Calculate \( \hat{Y} = \beta_0 + \beta_1 X \)
Calculate the predicted values and the residuals
Calculate the sums of squares, R² and SEE
Test the significance of \( \beta_1 \)

Step 1: Calculate the Linear Regression

We need to find \( \beta_0 \) (intercept) and \( \beta_1 \) (slope) of the line \( \hat{Y} = \beta_0 + \beta_1 X \).

Formulas:

\[ \beta_1 = \frac{n \sum XY - (\sum X)(\sum Y)}{n \sum X^2 - (\sum X)^2} \]

\[ \beta_0 = \bar{Y} - \beta_1 \bar{X} \]

Calculations:

Calculate the sums:
- \( \sum X = 2 + 4 + 6 + 8 + 10 = 30 \)
- \( \sum Y = 50 + 60 + 80 + 70 + 90 = 350 \)
- \( \sum XY = (2 \times 50) + (4 \times 60) + (6 \times 80) + (8 \times 70) + (10 \times 90) = 100 + 240 + 480 + 560 + 900 = 2280 \)
- \( \sum X^2 = 2^2 + 4^2 + 6^2 + 8^2 + 10^2 = 4 + 16 + 36 + 64 + 100 = 220 \)
Calculate \( \beta_1 \): \[ \beta_1 = \frac{5 \times 2280 - 30 \times 350}{5 \times 220 - 30^2} = \frac{11400 - 10500}{1100 - 900} = \frac{900}{200} = 4.5 \]
Calculate \( \beta_0 \): \[ \bar{X} = \frac{30}{5} = 6, \quad \bar{Y} = \frac{350}{5} = 70 \] \[ \beta_0 = 70 - 4.5 \times 6 = 70 - 27 = 43 \]
Regression equation: \[ \hat{Y} = 43 + 4.5X \]

Step 2: Calculate the Predicted Values and the Residuals

We apply the equation to each \( X \):

Student	\( X \)	\( Y \)	\( \hat{Y} = 43 + 4.5X \)	\( Y - \hat{Y} \) (residual)
1	2	50	\( 43 + 4.5 \times 2 = 52 \)	\( 50 - 52 = -2 \)
2	4	60	\( 43 + 4.5 \times 4 = 61 \)	\( 60 - 61 = -1 \)
3	6	80	\( 43 + 4.5 \times 6 = 70 \)	\( 80 - 70 = +10 \)
4	8	70	\( 43 + 4.5 \times 8 = 79 \)	\( 70 - 79 = -9 \)
5	10	90	\( 43 + 4.5 \times 10 = 88 \)	\( 90 - 88 = +2 \)

Notice that the residuals add up to zero: \( -2 - 1 + 10 - 9 + 2 = 0 \).

Step 3: Calculate the Sums of Squares

1. Total sum of squares (\( SS_{\text{Total}} \))

Measures the total variation of \( Y \) around its mean.

\[ SS_{\text{Total}} = \sum (Y_i - \bar{Y})^2 \]

\( Y_i \)	\( Y_i - \bar{Y} \)	\( (Y_i - \bar{Y})^2 \)
50	-20	400
60	-10	100
80	+10	100
70	0	0
90	+20	400

\[ SS_{\text{Total}} = 400 + 100 + 100 + 0 + 400 = 1000 \]

2. Regression sum of squares (\( SS_{\text{Reg}} \))

Measures how much of the variation of \( Y \) is explained by the model.

\[ SS_{\text{Reg}} = \sum (\hat{Y}_i - \bar{Y})^2 \]

\( \hat{Y}_i \)	\( \hat{Y}_i - \bar{Y} \)	\( (\hat{Y}_i - \bar{Y})^2 \)
52	-18	324
61	-9	81
70	0	0
79	+9	81
88	+18	324

\[ SS_{\text{Reg}} = 324 + 81 + 0 + 81 + 324 = 810 \]

3. Sum of squared residuals (\( SS_{\text{Res}} \) or SSE)

Measures the variation the model does not explain.

\[ SSE = \sum (Y_i - \hat{Y}_i)^2 \]

Residual (\( Y_i - \hat{Y}_i \))	\( (Y_i - \hat{Y}_i)^2 \)
-2	4
-1	1
+10	100
-9	81
+2	4

\[ SSE = 4 + 1 + 100 + 81 + 4 = 190 \]

Note that \( SS_{\text{Total}} = SS_{\text{Reg}} + SSE \): \( 1000 = 810 + 190 \).

Step 4: Calculate R² (Coefficient of Determination)

\[ R^2 = \frac{SS_{\text{Reg}}}{SS_{\text{Total}}} = \frac{810}{1000} = 0.81 \quad \text{or} \quad R^2 = 1 - \frac{190}{1000} = 0.81 \]

Interpretation: the model explains 81% of the variation in scores. The remaining 19% is unexplained (residuals). The correlation between hours and scores is \( r = \sqrt{0.81} = 0.9 \).

Step 5: Calculate the SEE (Standard Error of the Estimate)

\[ SEE = \sqrt{\frac{SSE}{n - 2}} = \sqrt{\frac{190}{5 - 2}} = \sqrt{63.33} \approx 7.96 \]

Interpretation: the model's predictions are typically off by about 8 points either way.

Step 6: Test the Significance of \( \beta_1 \)

First, the sum of squared deviations of X: \( \sum (x_i - \bar{x})^2 = 16 + 4 + 0 + 4 + 16 = 40 \).

\[ SE(\hat{\beta}_1) = \frac{7.96}{\sqrt{40}} \approx 1.258, \qquad t = \frac{4.5}{1.258} \approx 3.58 \]

With \( n - 2 = 3 \) degrees of freedom, the two-sided 5% critical value is \( t_{0.025;\,3} = 3.182 \). Since \( 3.58 > 3.182 \), we reject \( H_0 \): the p-value is about 0.037. The 95% confidence interval for the slope is:

\[ 4.5 \pm 3.182 \times 1.258 = [0.50,\ 8.50] \]

The interval does not include zero, which confirms the test's conclusion. Notice, however, how wide it is: with only 5 students, each extra hour of study could be worth anywhere from half a point to 8.5 points.

Summary of results

Regression equation: \( \hat{Y} = 43 + 4.5X \)
R² = 0.81 (81% of the variation explained)
SSE = 190 (sum of squared residuals)
SEE ≈ 7.96 (typical prediction error)
t ≈ 3.58, p ≈ 0.037 (slope significant at 5%)

In the study-hours example, the slope t-test gave t ≈ 3.58 with 3 degrees of freedom (critical value 3.182). Which conclusion is correct?

Answer: We reject H0: there is evidence, at the 5% level, that study hours are related to the score — 3.58 is greater than 3.182 (p ≈ 0.037). The third option is tempting, but regression shows association, not causation.

Exercises

1. A regression between age (X) and blood pressure (Y) has \( R^2 = 0.64 \). What does this mean?

a) 64% of the variation in Y is explained by X
b) The correlation between X and Y is 0.64
c) The model is invalid
d) 36% of the data are not explained by Y

2. The residuals form a U pattern. What does this indicate?

a) Homoscedasticity
b) Heteroscedasticity
c) A non-linear relationship that the model did not capture
d) A good regression

3. Given the table below, calculate the linear regression and interpret R² and SEE.

X (years of experience)	Y (salary)
1	30,000
3	40,000
5	38,000
7	55,000

a) Calculate \( \beta_0 \) and \( \beta_1 \).
b) Calculate \( R^2 \) and the SEE.

4. Run a regression with the data below and discuss whether the model is good.

X (advertising)	Y (sales)
100	80
200	70
300	90
350	110

Tip: calculate \( R^2 \), the SEE and plot the residuals!

5. Someone proposed the predictions in the table below for the scores (they do not come from the least-squares line). Calculate \( R^2 \) and the SEE and plot the residuals. Can we say these predictions are good? Check the skewness and kurtosis of the residuals.

Student	Hours (X)	Score (Y)	Prediction (\( \hat{Y} \))
1	2	50	55
2	4	60	65
3	6	80	75
4	8	70	85
5	10	90	95

Answer Key

1. a). Option b) is wrong because the correlation is the square root of R²: \( r = \sqrt{0.64} = 0.8 \).

2. c). A U in the residuals indicates a curved relationship; heteroscedasticity shows up as a funnel.

3. \( \beta_1 = 3650 \) and \( \beta_0 = 26150 \), so \( \hat{Y} = 26150 + 3650X \): each year of experience adds, on average, 3,650 to the salary. The residuals are 200, 2,900, −6,400 and 3,300; SSE = 60,300,000 and \( SS_{\text{Total}} \) = 326,750,000, so \( R^2 \approx 0.815 \) (81.5% of the variation in salaries is explained by experience) and \( SEE = \sqrt{60{,}300{,}000/2} \approx 5{,}491 \) (predictions are typically off by about 5.5 thousand). Be careful: with only 4 observations, the slope t-test gives p ≈ 0.097, so despite the high R² the relationship is not significant at 5%.

4. \( \hat{Y} \approx 59.32 + 0.1186X \); residuals ≈ 8.81, −13.05, −4.92 and 9.15; SSE ≈ 355.9; \( R^2 \approx 0.593 \); \( SEE \approx 13.34 \); slope p-value ≈ 0.23. The model is not good: it explains only 59% of the variation, the typical error (13 units) is large for sales between 70 and 110, the slope is not significant, and the residuals alternate in sign with no clear pattern, which with 4 points does not allow many conclusions.

5. Residuals: −5, −5, +5, −15, −5. SSE = 325, so \( R^2 = 1 - 325/1000 = 0.675 \) and \( SEE = \sqrt{325/3} \approx 10.41 \). The residuals have a mean of −5: the predictions systematically overestimate the scores (bias). The skewness of the residuals is 0 (they are symmetric around −5); the kurtosis, with only 5 points, is unreliable (−0.5 with the population formula, 2.0 with pandas' small-sample correction). The predictions are not good: the least-squares line from the example (R² = 0.81, SEE ≈ 7.96) is better. Notice also that here \( SS_{\text{Reg}}/SS_{\text{Total}} \) would give 1125/1000 = 1.125: the two R² formulas only agree for the least-squares line, and for other predictions you should use \( 1 - SSE/SS_{\text{Total}} \).

이 기사가 어떠셨나요?

0^%

사용자 평점 (0)

이 기사를 공유하세요!